Study on Acoustic Model Personalization in a Context of Collaborative Learning Constrained by Privacy Preservation

This paper investigates different approaches in order to improve the performance of a speech recognition system for given speaker by using no more than 5 min from this speaker, and without exchanging data other users/speakers. Inspired federated learning paradigm, we consider speakers that have access personalized database their own speech, learn an acoustic model collaborate with network model. Several local personalizations are explored depending on how aggregation mechanisms performed. We study impact selecting, adaptive way, subset speakers’s models based notion similarity. also investigate effect weighted averaging fine-tuned global models. In our approach, only neural parameters exchanged audio is exchanged. By avoiding communicating personal data, proposed approach tends preserve privacy speakers.Experiments conducted TEDLIUM 3 dataset show best improvement several user datasets. Our applied HMM/TDNN improves quickly significantly ASR terms WER (for instance one two evaluation datasets, 14.84% 13.45% less per speaker) .